Pandas Profiling Report

Dataset statistics

Number of variables	8
Number of observations	12838
Missing cells	116
Missing cells (%)	0.1%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	902.7 KiB
Average record size in memory	72.0 B

Variable types

Numeric	8

Alerts

`observedTempMax` is highly correlated with `observedTempMin` and 2 other fields	High correlation
`observedTempMin` is highly correlated with `observedTempMax` and 3 other fields	High correlation
`observedHumidity` is highly correlated with `observedTempMin` and 2 other fields	High correlation
`observedPressure` is highly correlated with `observedTempMax` and 3 other fields	High correlation
`observedRainfall` is highly correlated with `observedTempMax` and 3 other fields	High correlation
`observedTempMax` is highly correlated with `observedTempMin` and 1 other fields	High correlation
`observedTempMin` is highly correlated with `observedTempMax` and 3 other fields	High correlation
`observedHumidity` is highly correlated with `observedTempMin` and 2 other fields	High correlation
`observedPressure` is highly correlated with `observedTempMax` and 3 other fields	High correlation
`observedRainfall` is highly correlated with `observedTempMin` and 2 other fields	High correlation
`observedTempMax` is highly correlated with `observedTempMin`	High correlation
`observedTempMin` is highly correlated with `observedTempMax` and 2 other fields	High correlation
`observedHumidity` is highly correlated with `observedRainfall`	High correlation
`observedPressure` is highly correlated with `observedTempMin` and 1 other fields	High correlation
`observedRainfall` is highly correlated with `observedTempMin` and 2 other fields	High correlation
`stationID` is highly correlated with `observedWind`	High correlation
`Days` is highly correlated with `observedTempMax` and 5 other fields	High correlation
`observedTempMax` is highly correlated with `Days` and 4 other fields	High correlation
`observedTempMin` is highly correlated with `Days` and 4 other fields	High correlation
`observedHumidity` is highly correlated with `Days` and 4 other fields	High correlation
`observedPressure` is highly correlated with `Days` and 4 other fields	High correlation
`observedWind` is highly correlated with `stationID` and 1 other fields	High correlation
`observedRainfall` is highly correlated with `Days` and 4 other fields	High correlation
`observedRainfall` has 1042 (8.1%) zeros	Zeros

Reproduction

Analysis started	2022-06-03 17:12:51.008812
Analysis finished	2022-06-03 17:13:06.139221
Duration	15.13 seconds
Software version	pandas-profiling v3.1.0
Download configuration	config.json

stationID
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	36
Distinct (%)	0.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	15817.08599

Minimum	10120
Maximum	41977
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	200.6 KiB

Quantile statistics

Minimum	10120
5-th percentile	10208
Q1	11111
median	11706
Q3	11929
95-th percentile	41958
Maximum	41977
Range	31857
Interquartile range (IQR)	818

Descriptive statistics

Standard deviation	10755.02454
Coefficient of variation (CV)	0.6799624497
Kurtosis	2.05577409
Mean	15817.08599
Median Absolute Deviation (MAD)	390
Skewness	2.008270185
Sum	203059750
Variance	115670552.9
Monotonicity	Increasing

Histogram with fixed size bins (bins=36)

Value	Count	Frequency (%)
11805	366	2.9%
10208	366	2.9%
11505	366	2.9%
11513	366	2.9%
11809	366	2.9%
11921	366	2.9%
11929	366	2.9%
10120	366	2.9%
10320	366	2.9%
11925	366	2.9%
Other values (26)	9178	71.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
10120	366	2.9%
10208	366	2.9%
10320	366	2.9%
10408	366	2.9%
10609	366	2.9%
10705	366	2.9%
10724	366	2.9%
10910	366	2.9%
11111	366	2.9%
11313	366	2.9%

Value	Count	Frequency (%)
41977	366	2.9%
41958	366	2.9%
41947	28	0.2%
41926	366	2.9%
41909	366	2.9%
41858	366	2.9%
12110	366	2.9%
12103	366	2.9%
12007	366	2.9%
11929	366	2.9%

Days
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	366
Distinct (%)	2.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	183.1314068

Minimum	1
Maximum	366
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	200.6 KiB

Quantile statistics

Minimum	1
5-th percentile	18
Q1	91
median	183
Q3	275
95-th percentile	348
Maximum	366
Range	365
Interquartile range (IQR)	184

Descriptive statistics

Standard deviation	105.8382791
Coefficient of variation (CV)	0.5779362533
Kurtosis	-1.201771358
Mean	183.1314068
Median Absolute Deviation (MAD)	92
Skewness	0.00150696532
Sum	2351041
Variance	11201.74132
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
8	36	0.3%
1	36	0.3%
17	36	0.3%
25	36	0.3%
2	36	0.3%
10	36	0.3%
18	36	0.3%
26	36	0.3%
3	36	0.3%
11	36	0.3%
Other values (356)	12478	97.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	36	0.3%
2	36	0.3%
3	36	0.3%
4	36	0.3%
5	36	0.3%
6	36	0.3%
7	36	0.3%
8	36	0.3%
9	36	0.3%
10	36	0.3%

Value	Count	Frequency (%)
366	35	0.3%
365	35	0.3%
364	35	0.3%
363	35	0.3%
362	35	0.3%
361	35	0.3%
360	35	0.3%
359	35	0.3%
358	35	0.3%
357	35	0.3%

observedTempMax
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	9330
Distinct (%)	72.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	30.63729508

Minimum	21.42068966
Maximum	36.85483871
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	200.6 KiB

Quantile statistics

Minimum	21.42068966
5-th percentile	25.08679487
Q1	29.1493956
median	31.47266214
Q3	32.44566636
95-th percentile	34.09111888
Maximum	36.85483871
Range	15.43414905
Interquartile range (IQR)	3.296270752

Descriptive statistics

Standard deviation	2.752837476
Coefficient of variation (CV)	0.08985249735
Kurtosis	0.06089005398
Mean	30.63729508
Median Absolute Deviation (MAD)	1.252405732
Skewness	-0.8403224256
Sum	393321.5943
Variance	7.578114167
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
31.54871795	10	0.1%
30.85641026	9	0.1%
32.4	9	0.1%
31.85128205	9	0.1%
31.85641026	9	0.1%
32.2974359	9	0.1%
32.40512821	9	0.1%
31.72307692	9	0.1%
32.22820513	8	0.1%
32.23076923	8	0.1%
Other values (9320)	12749	99.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
21.42068966	1	< 0.1%
21.91794872	1	< 0.1%
21.96206897	1	< 0.1%
21.99655172	1	< 0.1%
22.03076923	1	< 0.1%
22.07241379	1	< 0.1%
22.09230769	1	< 0.1%
22.10512821	1	< 0.1%
22.11538462	1	< 0.1%
22.12413793	1	< 0.1%

Value	Count	Frequency (%)
36.85483871	1	< 0.1%
36.84193548	1	< 0.1%
36.78064516	2	< 0.1%
36.75384615	1	< 0.1%
36.67419355	1	< 0.1%
36.59487179	1	< 0.1%
36.55641026	1	< 0.1%
36.55128205	1	< 0.1%
36.53846154	2	< 0.1%
36.53548387	1	< 0.1%

observedTempMin
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	9535
Distinct (%)	74.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	21.32676368

Minimum	9.252631579
Maximum	27.17096774
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	200.6 KiB

Quantile statistics

Minimum	9.252631579
5-th percentile	12.15897436
Q1	17.04679487
median	23.53076923
Q3	25.51702381
95-th percentile	26.26451613
Maximum	27.17096774
Range	17.91833616
Interquartile range (IQR)	8.470228938

Descriptive statistics

Standard deviation	4.904167693
Coefficient of variation (CV)	0.2299536754
Kurtosis	-0.9072668793
Mean	21.32676368
Median Absolute Deviation (MAD)	2.487179487
Skewness	-0.723814616
Sum	273792.9921
Variance	24.05086076
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
26.1	17	0.1%
25.73076923	11	0.1%
26	10	0.1%
26.25384615	10	0.1%
25.96666667	10	0.1%
26.04871795	10	0.1%
25.5	10	0.1%
26.06153846	10	0.1%
25.91025641	10	0.1%
25.6	9	0.1%
Other values (9525)	12731	99.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
9.252631579	1	< 0.1%
9.3875	1	< 0.1%
9.486842105	1	< 0.1%
9.556410256	1	< 0.1%
9.55862069	1	< 0.1%
9.629411765	1	< 0.1%
9.634210526	1	< 0.1%
9.665789474	1	< 0.1%
9.679487179	1	< 0.1%
9.679487179	1	< 0.1%

Value	Count	Frequency (%)
27.17096774	1	< 0.1%
27.09677419	1	< 0.1%
27.01538462	1	< 0.1%
26.98717949	1	< 0.1%
26.96774194	1	< 0.1%
26.96451613	1	< 0.1%
26.95128205	1	< 0.1%
26.91794872	1	< 0.1%
26.87741935	1	< 0.1%
26.87692308	1	< 0.1%

observedHumidity
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	3781
Distinct (%)	29.5%
Missing	30
Missing (%)	0.2%
Infinite	0
Infinite (%)	0.0%
Mean	79.9061145

Minimum	56.69230769
Maximum	97
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	200.6 KiB

Quantile statistics

Minimum	56.69230769
5-th percentile	68.43113553
Q1	76.1025641
median	80.41025641
Q3	84.8974359
95-th percentile	88.35897436
Maximum	97
Range	40.30769231
Interquartile range (IQR)	8.794871795

Descriptive statistics

Standard deviation	6.13144073
Coefficient of variation (CV)	0.07673306065
Kurtosis	-0.02732654053
Mean	79.9061145
Median Absolute Deviation (MAD)	4.41025641
Skewness	-0.6059856807
Sum	1023437.514
Variance	37.59456542
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
85	51	0.4%
80	51	0.4%
79	50	0.4%
77	47	0.4%
81	44	0.3%
82	41	0.3%
78	37	0.3%
84	36	0.3%
76	36	0.3%
86	33	0.3%
Other values (3771)	12382	96.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
56.69230769	1	< 0.1%
56.94871795	1	< 0.1%
57.15384615	1	< 0.1%
58.15384615	1	< 0.1%
58.30769231	1	< 0.1%
58.43589744	1	< 0.1%
58.92307692	1	< 0.1%
58.97435897	1	< 0.1%
59.02564103	2	< 0.1%
59.12820513	1	< 0.1%

Value	Count	Frequency (%)
97	1	< 0.1%
96	2	< 0.1%
95	4	< 0.1%
94	6	< 0.1%
93	3	< 0.1%
92	7	0.1%
91.66666667	1	< 0.1%
91.56410256	1	< 0.1%
91.47368421	1	< 0.1%
91.35897436	1	< 0.1%

observedPressure
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	9485
Distinct (%)	74.0%
Missing	28
Missing (%)	0.2%
Infinite	0
Infinite (%)	0.0%
Mean	1008.018912

Minimum	974.8538462
Maximum	1039.406897
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	200.6 KiB

Quantile statistics

Minimum	974.8538462
5-th percentile	1000.272073
Q1	1003.298077
median	1008.169231
Q3	1012.775827
95-th percentile	1015.285985
Maximum	1039.406897
Range	64.5530504
Interquartile range (IQR)	9.477749663

Descriptive statistics

Standard deviation	5.452238648
Coefficient of variation (CV)	0.005408865431
Kurtosis	1.548732333
Mean	1008.018912
Median Absolute Deviation (MAD)	4.730952381
Skewness	-0.5569541696
Sum	12912722.27
Variance	29.72690627
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1012	11	0.1%
1000.4	11	0.1%
1015.3	11	0.1%
1015.1	8	0.1%
1012.7	8	0.1%
1004.8	8	0.1%
1003.5	8	0.1%
1007.9	8	0.1%
1003.082051	7	0.1%
1014.9	7	0.1%
Other values (9475)	12723	99.1%
(Missing)	28	0.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
974.8538462	1	< 0.1%
975.8538462	1	< 0.1%
976.9897436	1	< 0.1%
979	1	< 0.1%
979.3137931	1	< 0.1%
979.462069	1	< 0.1%
979.5793103	1	< 0.1%
979.7206897	1	< 0.1%
979.8172414	1	< 0.1%
979.8448276	1	< 0.1%

Value	Count	Frequency (%)
1039.406897	1	< 0.1%
1038.913793	1	< 0.1%
1037.212121	1	< 0.1%
1034.173684	1	< 0.1%
1033.146154	1	< 0.1%
1029.371053	1	< 0.1%
1023.255172	1	< 0.1%
1018.8	2	< 0.1%
1018.430769	1	< 0.1%
1018.2	1	< 0.1%

observedWind
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	6503
Distinct (%)	50.8%
Missing	30
Missing (%)	0.2%
Infinite	0
Infinite (%)	0.0%
Mean	3.202190545

Minimum	0
Maximum	12.2
Zeros	4
Zeros (%)	< 0.1%
Negative	0
Negative (%)	0.0%
Memory size	200.6 KiB

Quantile statistics

Minimum	0
5-th percentile	1.607692308
Q1	2.279411765
median	2.984615385
Q3	3.77742915
95-th percentile	5.664102564
Maximum	12.2
Range	12.2
Interquartile range (IQR)	1.498017385

Descriptive statistics

Standard deviation	1.336709934
Coefficient of variation (CV)	0.4174361005
Kurtosis	4.06988502
Mean	3.202190545
Median Absolute Deviation (MAD)	0.7435897436
Skewness	1.591977538
Sum	41013.65651
Variance	1.786793449
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
2	82	0.6%
3	48	0.4%
4	39	0.3%
5	33	0.3%
2.5	32	0.2%
2.3	20	0.2%
3.3	20	0.2%
3.5	20	0.2%
2.2	18	0.1%
3.2	18	0.1%
Other values (6493)	12478	97.2%
(Missing)	30	0.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	4	< 0.1%
0.2857142857	1	< 0.1%
0.5820512821	1	< 0.1%
0.5846153846	1	< 0.1%
0.7552631579	1	< 0.1%
0.7702702703	1	< 0.1%
0.8459459459	1	< 0.1%
0.8461538462	1	< 0.1%
0.8473684211	1	< 0.1%
0.8594594595	1	< 0.1%

Value	Count	Frequency (%)
12.2	1	< 0.1%
11.8	1	< 0.1%
10.4	1	< 0.1%
10.35294118	1	< 0.1%
10.1	1	< 0.1%
10.08529412	1	< 0.1%
10.04705882	1	< 0.1%
10.02941176	1	< 0.1%
10.01764706	1	< 0.1%
10	1	< 0.1%

observedRainfall
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
ZEROS

Distinct	3351
Distinct (%)	26.2%
Missing	28
Missing (%)	0.2%
Infinite	0
Infinite (%)	0.0%
Mean	6.70611249

Minimum	0
Maximum	57.1025641
Zeros	1042
Zeros (%)	8.1%
Negative	0
Negative (%)	0.0%
Memory size	200.6 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0.5128205128
median	4.124708625
Q3	10.56410256
95-th percentile	22.03974359
Maximum	57.1025641
Range	57.1025641
Interquartile range (IQR)	10.05128205

Descriptive statistics

Standard deviation	7.661600358
Coefficient of variation (CV)	1.142480143
Kurtosis	3.269712888
Mean	6.70611249
Median Absolute Deviation (MAD)	3.970862471
Skewness	1.620469935
Sum	85905.30099
Variance	58.70012004
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	1042	8.1%
0.02564102564	180	1.4%
0.05128205128	145	1.1%
0.07692307692	110	0.9%
0.1025641026	97	0.8%
0.1794871795	80	0.6%
0.1538461538	76	0.6%
0.3333333333	71	0.6%
0.2307692308	65	0.5%
0.1282051282	64	0.5%
Other values (3341)	10880	84.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1042	8.1%
0.01282051282	1	< 0.1%
0.02564102564	180	1.4%
0.02631578947	30	0.2%
0.02702702703	4	< 0.1%
0.02777777778	6	< 0.1%
0.02857142857	12	0.1%
0.02941176471	6	< 0.1%
0.0303030303	10	0.1%
0.03225806452	8	0.1%

Value	Count	Frequency (%)
57.1025641	1	< 0.1%
51.12820513	1	< 0.1%
50.92307692	1	< 0.1%
50.74285714	1	< 0.1%
49.20512821	1	< 0.1%
49.00641026	1	< 0.1%
48.61538462	1	< 0.1%
47.97435897	1	< 0.1%
47.48571429	1	< 0.1%
47.25641026	1	< 0.1%

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The correlation heatmap measures nullity correlation: how strongly the presence or absence of one variable affects the presence of another.

The dendrogram allows you to more fully correlate variable completion, revealing trends deeper than the pairwise ones visible in the correlation heatmap.

First rows

	stationID	Days	observedTempMax	observedTempMin	observedHumidity	observedPressure	observedWind	observedRainfall
0	10120	1	23.182051	10.735897	77.586207	1016.255172	3.968966	0.034483
1	10120	2	22.997436	10.738462	78.655172	1016.424138	4.017241	0.310345
2	10120	3	22.276923	10.566667	80.551724	1016.227586	4.306897	0.448276
3	10120	4	22.115385	10.712821	79.862069	1016.027586	4.234483	0.758621
4	10120	5	22.325641	10.274359	79.758621	1015.627586	4.434483	0.034483
5	10120	6	22.648718	10.028205	79.000000	1015.765517	3.410345	0.000000
6	10120	7	22.512821	10.310256	79.655172	1016.141379	3.972414	0.000000
7	10120	8	22.533333	10.182051	78.931034	1015.827586	3.806897	0.827586
8	10120	9	22.092308	10.494872	80.034483	1015.786207	3.679310	0.655172
9	10120	10	21.917949	9.892308	79.413793	1015.903448	3.558621	0.000000

Last rows

	stationID	Days	observedTempMax	observedTempMin	observedHumidity	observedPressure	observedWind	observedRainfall
12828	41977	357	26.904762	15.147619	NaN	NaN	NaN	NaN
12829	41977	358	27.409524	15.380952	NaN	NaN	NaN	NaN
12830	41977	359	27.304762	15.228571	NaN	NaN	NaN	NaN
12831	41977	360	27.100000	14.938095	NaN	NaN	NaN	NaN
12832	41977	361	26.733333	15.042857	NaN	NaN	NaN	NaN
12833	41977	362	26.633333	14.795238	NaN	NaN	NaN	NaN
12834	41977	363	27.047619	14.319048	NaN	NaN	NaN	NaN
12835	41977	364	27.414286	14.376190	NaN	NaN	NaN	NaN
12836	41977	365	27.314286	14.476190	NaN	NaN	NaN	NaN
12837	41977	366	26.900000	13.620000	NaN	NaN	NaN	NaN

Overview

Variables

Interactions

Correlations

Spearman's ρ

Pearson's r

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows